Towards a Computer-Based Understanding of Medieval Images

Research in the field of cultural heritage requires computer vision algorithms that can automatically advance to the representational content of images. To make large scale image databases accessible it is crucial that computer-based object retrieval in images lives up to what it really is, a search through images rather than a search through textual annotations as in many current retrieval systems in cultural heritage. Our contribution is threefold: (1) Benchmarking: We have assembled a novel image dataset of medieval images with groundtruth information. Its completeness and scientific significance in the humanities allows, for the first time, to thoroughly evaluate retrieval algorithms for cultural heritage. (2) Object analysis: Object shape is automatically extracted from tinted drawings and we present a statistical algorithm that automatically analyzes the type-variability of object categories. The discovered category substructure is, in its richness, not captured by the currently used annotations. (3) Recognition: A category-level retrieval system is presented that detects objects in images and, thus, provides object locations which are not available in current metadata.